Statistical Analyses for measuring the impact of COVID-19 Vaccine Misinformation on COVID-19 Vaccine Acceptance¶
This notebook covers all statistical models that generate results and figures in the main paper and supplementary. Transformed survey data are used.
We use pystan package, which is the Python implementation of the Stan platform for statistical modeling. This package allows us to perform full Bayesian inference using Hamiltonian Monte Carlo via No-U-Turn Sampler (NUTS).
For brevity, the analysis in this notebook looks at the UK, and can be easily repeated for the US. We begin by importing the processed data for the UK (see import_data.ipynb for more details). The src/utils.py file provides many helper functions which we will frequently use to generate tables and figures for the paper. All models in this notebook can be found in src/models.py but have been written out here to aid model description.
%matplotlib inline
import src.utils as ut
NUM_SAMPLES = 200
#here we have set samples to just 200 (x 4 chains), but in the paper we use 2000 samples (x 4 chains) for more credible estimates
df, dd = ut.import_transformed_data('dat/orb_uk')
Impact of Exposure to Misinformation¶
We begin by modeling the impact of exposure of (mis)information on COVID-19 vaccine acceptance. As noted in the paper, vaccine acceptance is a 4-class ordinal variable going from "Yes, definitely" to "No, definitely not". See the "Statistical Methods" section of the paper; specific model specs will be referred to in the code below.
Below is a function that essentially defines the corresponding Stan model, and fits it with the given data.
def model_impact(df, group=1., kind='self', prior_mu=1., prior_sigma=1., iters=NUM_SAMPLES):
# Model: Ref 1, Table 3
# Results: Table 2
import pystan as st
model_code = '''
data {
int<lower=0> n; //number of data points
int<lower=1> m; //number of conditions
int<lower=2> k; //number of outcomes
int<lower=1,upper=k> y[n,m]; //outcome per sample per condition
}
parameters {
real mu[k-1];
real<lower=0> sigma[k-1];
ordered [k-1] alpha[m];
}
model {
for (i in 1:(k-1)) {
mu[i] ~ normal(0, %f);
sigma[i] ~ exponential(%f);
}
for (i in 1:m)
alpha[i] ~ normal(mu, sigma);
for (i in 1:n)
for (j in 1:m)
y[i,j] ~ ordered_logistic(0, alpha[j]);
}
'''%(prior_mu, prior_sigma)
df = df.loc[df['Treatment']==group]
data = {'n':df.shape[0], 'm':2, 'k':4, 'y':df[['Vaccine Intent for %s (Pre)'%kind, 'Vaccine Intent for %s (Post)'%kind]].values}
model = st.StanModel(model_code=model_code)
fit = model.sampling(data=data, iter=iters)
return fit
Let's call the above function to generate the fit object, when modeling impact of exposure to misinformation (treatment group).
fit_impact_T = model_impact(df, group=1)
This object's plot() and print() methods allow us to see a quick summary of the model parameters. In particular, we are looking for well-mixed chains in the trace plots, and the Rhat statistic to be close to 1.
fit_impact_T.plot()
print(fit_impact_T)
A clear distinction of the $\alpha$ corresponding to pre- and post-exposure is visible in the third plot, which would indicate differences after misinformation exposure. We can use some helper functions to generate the posterior statistics---parameter mean to judge effect size and 95% percentile intervals (PI) to judge "significance"---of the distribution across the 4 vaccine acceptance categories from this fit object. This contributes to Table 2 of the paper.
fit2stats_impact_T = ut.stats_impact(fit_impact_T)
print(fit2stats_impact_T)
Looking at the difference of post and pre exposures, we see a significant (PI excludes 0) drop in people who would "definitely" accept the vaccine, and a significant rise in those who are unsure but leaning towards no, or definitely will not accept the vaccine. Similarly, we can repeat this analysis for exposure to factual information (control group).
fit_impact_C = model_impact(df, group=0)
fit2stats_impact_C = ut.stats_impact(fit_impact_C)
print(fit2stats_impact_C)
Here, we do not see a significant change in the distribution of respondents across any vaccine acceptance category (PI includes 0).
Causal Risk Difference¶
However, as demonstrated in import_data.ipynb, it's not the case that people do not change their vaccine opinions at all. This can be due to recall-bias, or other individual-effects, which "cancel" out in a manner that the aggregate distribution in the society remains the same. This can be better understood by computing a "risk-difference" of exposure to misinformation over exposure to factual-information---see Equation 2 of the paper.
def model_impact_causal(df, kind='self', prior_mu=1., prior_sigma=1., prior_rho=1., iters=NUM_SAMPLES):
# Model: Appendix C
# Results: Tables S1, S2; Figures S1, S2
import pystan as st
model_code = '''
data {
int<lower=0> n; //number of data points
int<lower=1> m; //number of conditions
int<lower=2> k; //number of outcomes
int<lower=1,upper=m> x_cond[n]; //treatment group
int<lower=1,upper=k> y_pre[n]; //pre-exposure outcome
int<lower=1,upper=k> y_post[n]; //post-exposure outcome
}
parameters {
real mu_alpha[k-1];
real mu_beta;
real<lower=0> sigma_alpha[k-1];
real<lower=0> sigma_beta;
real<lower=0> sigma_delta;
simplex[k-1] delta_delta;
simplex[k-1] delta[m];
real beta[m];
ordered[k-1] alpha[m];
}
model {
mu_alpha ~ normal(0, %f);
mu_beta ~ normal(0, %f);
sigma_alpha ~ exponential(%f);
sigma_beta ~ exponential(%f);
sigma_delta ~ exponential(%f);
{
vector[k-1] u;
for (i in 1:(k-1))
u[i] = 1;
delta_delta ~ dirichlet(%f*u);
}
for (i in 1:m){
beta[i] ~ normal(mu_beta, sigma_beta);
alpha[i] ~ normal(mu_alpha, sigma_alpha);
delta[i] ~ dirichlet(sigma_delta*delta_delta);
}
for (i in 1:n)
y_post[i] ~ ordered_logistic(beta[x_cond[i]]*sum(delta[x_cond[i]][:y_pre[i]-1]), alpha[x_cond[i]]);
}
'''%(prior_mu, prior_mu, prior_sigma, prior_sigma, prior_sigma, prior_rho)
data = {'n':df.shape[0], 'm':2, 'k':4, 'x_cond':df['Treatment'].values+1,
'y_pre':df['Vaccine Intent for %s (Pre)'%kind].values,
'y_post':df['Vaccine Intent for %s (Post)'%kind].values}
model = st.StanModel(model_code=model_code)
fit = model.sampling(data=data, iter=iters)
return fit
fit_impact_causal = model_impact_causal(df)
fit2stats_impact_causal = ut.stats_impact_causal(fit_impact_causal)
ut.unstack_df(fit2stats_impact_causal)
The above contributes to Tables S1, S2 and Figures S1, S2 of the paper. Let's generate those figures for a simple visual interpretation by using a helper function.
ut.plot_stats(fit2stats_impact_causal.loc['Treatment-Control']*100, oddsratio=False, title='$\\Delta_{RD}$ for UK', xlab='% Change between Treatment and Control', tick_suffix=' [POST]', label_suffix='\n[PRE]', factor=0.5)
It's evident that overall, respondents tend to transition from higher vaccine acceptance categoris to lower ones upon exposure to misinformation, relative to exposure to factual-information. An alternate representation of the entire above analysis is to draw Sankey plots which show the "flow" of people pre- and post-exposure to information.
ut.plot_causal_flow(fit2stats_impact_causal, fit2stats_impact_T, fit2stats_impact_C)
Determinants of Vaccine Hesitancy and Susceptibility to Misinformation¶
Once we have established that misinformation does impact vaccine acceptance, we can look at the socio-econo-demographic determinants of this hesitancy, and if certain groups of people might be more susceptible to COVID-19 vaccine misinformation.
To that end, we define a model which will determine the contribution of any variable to vaccine intent, while controling for the socio-demographic information.
def model_socdem(df, dd, atts=[], group=None, kind='self', prior_beta=1., prior_delta=1., prior_alpha=1., iters=NUM_SAMPLES):
# Model: Ref 2, 3, 4, 5 in Table 3
# Results: Tables S3, S4, S5; Figures 3, 4
import pystan as st
import numpy as np
from src.bayesoc import Dim #we define some helper classes to extract posterior samples easily
cats = ['Age', 'Gender', 'Education', 'Employment', 'Religion', 'Political', 'Ethnicity', 'Income']
if isinstance(atts, str): atts = [atts]
for att in atts: cats += [x for x in list(df) if x[:len(att)]==att]
outs = ['Vaccine Intent for self (Pre)', 'Vaccine Intent for self (Post)', 'Treatment']
df = df[cats+outs].dropna()
dims = [Dim(pi=len(dd[cat]), beta_prior=prior_beta, value=dd[cat].keys(), name=cat) for cat in cats]
stan = [d.get_stan() for d in dims]
code = {'data':[], 'parameters':[], 'model':[], 'output':[]}
for key in code:
for d in stan: code[key].append(d[key])
mod_cd = '''
data {
int<lower=1> n; //number of data points
int<lower=2> k; //number of outcomes
int<lower=1,upper=k> y_pre[n]; //pre-exposure
int<lower=1,upper=k> y_post[n]; //post-exposure
%s
}
parameters {
%s
simplex[k-1] delta;
ordered[k-1] alpha;
}
model {
%s
{
vector[k-1] u;
for (i in 1:(k-1))
u[i] = 1;
delta ~ dirichlet(%f*u);
}
alpha ~ normal(0, %f);
for (i in 1:n)
y_post[i] ~ ordered_logistic((%s)*sum(delta[:y_pre[i]-1]), alpha);
}
'''%('\n'.join(code['data']), '\n'.join(code['parameters']), '\n'.join(code['model']), prior_delta, prior_alpha, ' + '.join(code['output']))
mod_bs = '''
data {
int<lower=1> n; //number of data points
int<lower=2> k; //number of outcomes
int<lower=1,upper=k> y_pre[n]; //pre-exposure
%s
}
parameters {
%s
ordered[k-1] alpha;
}
model {
%s
alpha ~ normal(0, %f);
for (i in 1:n)
y_pre[i] ~ ordered_logistic(%s, alpha);
}
'''%('\n'.join(code['data']), '\n'.join(code['parameters']), '\n'.join(code['model']), prior_alpha, ' + '.join(code['output']))
data = {}
if group is not None:
df = df.loc[df['Treatment']==group]
data['y_post'] = df['Vaccine Intent for %s (Post)'%kind].values
data['n'] = df.shape[0]
data['k'] = 4
data['y_pre'] = df['Vaccine Intent for %s (Pre)'%kind].values
print('Dataframe of size:', df.shape)
for i in range(len(cats)):
name = dims[i].name
data['k_%s'%name] = len(dd[cats[i]])
data[name] = np.array(df[cats[i]].values, dtype=int)
if data[name].min()==0: data[name] += 1
if group is None: model = st.StanModel(model_code=mod_bs)
else: model = st.StanModel(model_code=mod_cd)
fit = model.sampling(data=data, iter=iters)
return fit
Socio-demographics and Social Media Usage¶
First, let us explore how social media usage might contribute to pre-exposure vaccine acceptance. This means a model with the socio-demographic as predictors, alongside social media usage variable, with the outcome being the pre-exposure vaccine intent. As before, we fit the model first.
fit_socdem_preexposure = model_socdem(df, dd, 'Social media')
Next, we extract the posterior statistics for the odds-ratio (OR) of different parameters that we have considered---all socio-demographics, plus social media usage. This contributes to Table S3 of the paper.
stats_socdem_preexposure = ut.stats_socdem(fit_socdem_preexposure, dd, df, 'Social media', oddsratio=True)
stats_socdem_preexposure
Next, we do a regression with the same predictors, except by adding the pre-exposure vaccine intent as an additional predictor with the outcome now being the post-exposure vaccine intent. This allows us to find determinants of "susceptibility" to misinformation. This contributes to Table S3 of the paper.
fit_socdem_susceptibility = model_socdem(df, dd, 'Social media', group=1)
stats_socdem_susceptibility = ut.stats_socdem(fit_socdem_susceptibility, dd, df, 'Social media', group=1, oddsratio=True)
Instead of printing out the whole table of results, we can use the plotting helper function to do a quick visualisation. This contributes to Figure 3 of the paper.
ut.plot_stats(stats_socdem_preexposure, stats_socdem_susceptibility, demos=True, title='Socio-demographic Determinants in UK', title_l='Pre-Exposure', title_r='Susceptibility', xlab='Odds Ratio', factor=0.5, titlesize=22, title_loc=-0.075)
We obtain some interesting results for the UK:
- Older age groups (55+) are significantly more willing to accept the vaccine than 18-24 year olds (OR<1).
- Females are significantly less willing to accept the vaccine than males (OR>1).
- Vaccine acceptance tends to increase with increase in education levels (OR significantly decreases w.r.t. baseline education Level-4). Something similar is observed with increase in income levels.
- Those who are retired are significantly more accepting of the vaccine when compared to those who are employed (OR<1).
- Those who do not support any mainstream political party in the UK, i.e. support "Others", are significantly less likely to accept the vaccine than compared to Conservatives (OR>1).
- Ethnic minorities are significantly less willing to accept a vaccine when compared to Whites (OR>1).
- Social media usage has no impact on vaccine acceptance (when controlling for all of the above socio-demographics).
Source of COVID-19 Information that are Trusted¶
It has been shown that trust plays a very important role when it comes to accessing public health information. We can thus do a similar analysis to observe how having trust in different sources of information for COVID-19 may determine vaccine hesitancy and susceptibility to misinformation.
fit_trust_preexposure = model_socdem(df, dd, 'Trust:')
fit_trust_susceptibility = model_socdem(df, dd, 'Trust:', group=1)
Next, we compute the posterior statistics. This contributes to Table S5 of the paper. This time, we will compute the log of ORs, since +/- log OR values make for easier interpretation in terms of ranking determinants by the magnitude of log ORs, than judging by >1/<1 for OR values.
stats_trust_preexposure = ut.stats_socdem(fit_trust_preexposure, dd, df, 'Trust:', oddsratio=False)
stats_trust_susceptibility = ut.stats_socdem(fit_trust_susceptibility, dd, df, 'Trust:', group=1, oddsratio=False)
We can use the same helper function to plot the statistics for us. This contributes to Figure 4 of the paper.
ut.plot_stats(stats_trust_preexposure, stats_trust_susceptibility, title='Sources of COVID-19 Info Trusted in UK', title_l='Pre-Exposure', title_r='Susceptibility', oddsratio=False, ylabel=False, xlab='Log Odds Ratio', factor=0.5, titlesize=20)
Clearly, the largest determinant of pre-exposure vaccine acceptance is not trusting any of the mainstream sources of information. Although this pool of people isn't huge (see the sample counts on the right-hand axis of the plot), they are significantly less likely to accept the vaccine than those who trust some conventional source of information. This group of people was also more susceptible to misinformation.
Whereas those who trusted TV news, government briefings, health authorities, and perhaps surprisingly celebrities, were more willing to accept the vaccine. Those indicating trust in Family and Friends were significantly more unwilling to accept the vaccine, which could indicate that receicing information from informal sources does not fare positively for COVID-19 vaccine acceptance.
Reasons for Vaccine Hesitancy¶
Those who did not indicate that they will "definitely" accept the vaccine were asked for their reasons to be hesitant. This can provide some insight into psycho-social determinants of hesitancy and susceptibility.
fit_reason_preexposure = model_socdem(df, dd, 'Reason:')
fit_reason_susceptibility = model_socdem(df, dd, 'Reason:', group=1)
Next, we compute the posterior statistics and visualise them, which contributes to Table S4 and Figure 4 of the paper.
stats_reason_preexposure = ut.stats_socdem(fit_reason_preexposure, dd, df, 'Reason:', oddsratio=False)
stats_reason_susceptibility = ut.stats_socdem(fit_reason_susceptibility, dd, df, 'Reason:', group=1, oddsratio=False)
ut.plot_stats(stats_reason_preexposure, stats_reason_susceptibility, title='Reasons for COVID-19 Vaccine Hesitancy in UK', title_l='Pre-Exposure', title_r='Susceptibility', oddsratio=False, ylabel=False, xlab='Log Odds Ratio', factor=0.5, titlesize=20, title_loc=0.05)
Evidently, only the group of people indicating they will "wait until others" received the vaccine, were significantly on the lower-end of COVID-19 vaccine hesitancy. Those indicating they were "not at risk" or "won't be ill" probably believe the risk of accepting the vaccine outweigh its need, and thus are also less willing to accept it. Those indicating worries about vaccine safety are also more unsure about the vaccine. But the largest contribution is those indicating "other" reasons, which do not fall into these "expected" reasons of hesitancy.
In terms of susceptibility, it seems like the only group that is significantly more susceptible is the one which indicated that the vaccine "approval may be rushed". This hints at the kind of psychological effect exposure to the misinformation images may have had on the respondents, perhaps provoking that fear of rushed approval.
What makes misinformation impactful?¶
After exposure to the 5 pieces of misinformation (treatment) or factual information (control), the respondents were asked 5 follow-up questions to judge their perception of each of the images they were shown. These questions asked them to rate on a 5-level scale the extent that: 1) they agree with the information displayed; 2) they are inclined to be vaccinated; 3) they believe the information to be trustworthy; 4) they will fact-check the information; and 5) they would share the image.
- The image makes them more inclined to get vaccinationed: "Vaccine Intent"
- They agree with the information displayed: "Agree with"
- They believe the information to be trustworthy: "Have trust in"
- They will fact-check the information: "Will fact-check"
- They will share the image: "Will share"
Let us infer the respondent's perceptions of each image they were shown.
def model_image_perceptions(df, group=1, prior_alpha=1., iters=NUM_SAMPLES):
# Model: Model for self-reported image-metrics
# Results: Figure 5
import pystan as st
import numpy as np
model_code = '''
data {
int<lower=0> n; //number of data points
int<lower=2> k; //number of outcomes
int<lower=1,upper=k> y[n]; //outcome per sample
}
parameters {
ordered [k-1] alpha;
}
model {
alpha ~ normal(0, %f);
for (i in 1:n)
y[i] ~ ordered_logistic(0, alpha);
}
'''%(prior_alpha)
metrics = ['Vaccine Intent', 'Agreement', 'Trust', 'Fact-check', 'Share']
fits = [dict() for i in range(5)]
df = df.loc[df['Treatment']==group]
for i in range(5):
for m in metrics:
data = {'n':df.shape[0], 'k':5, 'y':df['Image %i:%s'%(i+1, m)].values+3}
model = st.StanModel(model_code=model_code)
fits[i][m] = model.sampling(data=data, iter=iters)
return fits
fit_image_perceptions_T = model_image_perceptions(df, group=1)
fit_image_perceptions_C = model_image_perceptions(df, group=0)
As before, we now extract the posterior statistics. We can visualise the mean estimate of these 5-level Likert categories for every image and image-metric using a helper function. This contributes to Figure 5 of the paper.
stats_image_perceptions_T = ut.stats_image_perceptions(fit_image_perceptions_T)
stats_image_perceptions_C = ut.stats_image_perceptions(fit_image_perceptions_C)
ut.plot_image_perceptions([stats_image_perceptions_T, stats_image_perceptions_C], ['Treatment', 'Control'])
Evidently, upon seeing the misinformation images (treatment), people's "self-reported" vaccine intent lowers more than it rises (red bars longer than blue ones), and we note the opposite effect for factual images (control). This is interesting, because as noted in the first set of models, while there is indeed the measured drop in vaccine intent upon seeing misinformation, there was no aggregate measured change upon seeing the factual information, even though people "self-report" otherwise. For all other image-metrics, it is evident that people are more likely to agree with have trust in, and share the factual information; and less likely to fact-check it, when compared to misinformation.
Which images are more impactful than others?¶
One interesting follow-up analysis is to use the self-reported image metrics as "features" of the images, to compute the contribution every image has in lowering the measured vaccine intent.
def model_image_impact(df, group=1, kind='self', prior_beta=1., prior_delta=1., prior_gamma=1., prior_alpha=1., iters=NUM_SAMPLES):
# Model: Ref 6, Table 3
# Results: Tables S6, S7
import pystan as st
import numpy as np
model_code = '''
data {
int<lower=1> n; //number of data points
int<lower=1> p; //number of images
int<lower=1> m; //number of metrics
int<lower=2> k; //number of outcomes
int<lower=1,upper=k> y_pre[n]; //pre-exposure
int<lower=1,upper=k> y_post[n]; //post-exposure
matrix[p,m] x_img[n]; //image metrics
}
parameters {
vector[m] beta;
simplex[p] gamma;
simplex[k-1] delta;
ordered[k-1] alpha;
}
model {
beta ~ normal(0, %f);
{
vector[p] u_img;
for (i in 1:p)
u_img[i] = 1;
gamma ~ dirichlet(%f*u_img);
}
{
vector[k-1] u;
for (i in 1:(k-1))
u[i] = 1;
delta ~ dirichlet(%f*u);
}
alpha ~ normal(0, %f);
for (i in 1:n)
{
real b = to_row_vector(gamma)*x_img[i]*beta;
y_post[i] ~ ordered_logistic(b*sum(delta[:y_pre[i]-1]), alpha);
}
}
'''%(prior_beta, prior_gamma, prior_delta, prior_alpha)
metrics = ['Vaccine Intent', 'Agreement', 'Trust', 'Fact-check', 'Share']
df = df.loc[df['Treatment']==group]
x = np.dstack([df[['Image %i:%s'%(i+1, m) for i in range(5)]].values for m in metrics])
data = {'n':df.shape[0], 'p':5, 'm':len(metrics), 'k':4, 'x_img':x,
'y_pre':df['Vaccine Intent for %s (Pre)'%kind].values,
'y_post':df['Vaccine Intent for %s (Post)'%kind].values}
model = st.StanModel(model_code=model_code)
fit = model.sampling(data=data, iter=iters)
return fit
fit_image_impact_T = model_image_impact(df, group=1)
fit_image_impact_C = model_image_impact(df, group=0)
As before, we now estimate posterior statistics on $\beta$, which refers to the contribution of every image-metric, and $\gamma$, which measures the weights of every image. This contributes to Tables S6, S7 in the paper.
stats_image_impact_T = ut.stats_image_impact(fit_image_impact_T, oddsratio=True)
stats_image_impact_C = ut.stats_image_impact(fit_image_impact_C, oddsratio=True)
ut.combine_dfs(stats_image_impact_T, stats_image_impact_C, '(Treatment)', '(Control)')
Observing $\gamma$, it appears that the first/fourth image in the misinformation set has the highest/lowest impact on reducing acceptance of a COVID-19 vaccine. Whereas the third image in the factual-information set has the lowest impact on reducing vaccine acceptance. A qualitative assessment of these images makes it clear that they have a scientific temperament, in terms of the semantic content. This could be indicative of scientific messaging having a bigger role to play than "memetic" information in swaying COVID-19 vaccine opinion.
Does misinformation spiral into filter bubbles?¶
This study shows that their is a measurable impact of COVID-19 vaccine misinformation on accepting a potential COVID-19 vaccine. However being a survey study, it has many limitations. In particular, this study doesn't capture the complex ways in which information is exposed to people on social media platforms---governed by a complex combination of what the platform's algorithms show them, what theior freidns or followers share with them, and what they choose to consume. Therefore, we can expect the effects to amplify (or dilute down) in a natural setting. However, given that information selection in an online setting tends to cause people to view more of the content that aligns with their ideologies, it often leads to a positive feedback loop wherein "filter bubbles" can form. We wanted to investigate if there is some evidence of such effects with regards to COVID-19 vaccine misinformation and acceptance.
One may hypothesise that those who are less accepting of the vaccine may have been exposed to more misinformation, and less of the factual information about them. After the exposure, we asked respondents if they had seen "similar" content circulating on the social media platforms that they used recently, in the last one month---to which they could respond with "yes" or "no". Based on that response, we can test the above hypothesis.
def model_filterbubble(df, group=1, kind='self', prior_beta=1., prior_alpha=1., iters=NUM_SAMPLES):
# Model: Model for evidence of filter-bubble effects of (mis)information exposure with regards to vaccine intent
# Results: Table S8; Figure S3
import pystan as st
import numpy as np
model_code = '''
data {
int<lower=1> n; //number of data points
int<lower=1> m; //number of categories
int<lower=2> k; //number of outcomes
int<lower=1,upper=m> y_pre[n]; //pre-exposure
int<lower=1,upper=k> y_see[n]; //seen images
}
parameters {
real beta[m];
ordered[k-1] alpha;
}
model {
beta ~ normal(0, %f);
alpha ~ normal(0, %f);
for (i in 1:n)
y_see[i] ~ ordered_logistic(beta[y_pre[i]], alpha);
}
'''%(prior_beta, prior_alpha)
df = df.loc[(df['Treatment']==group) & (df['Seen such online content']!=3)] #ignoring do-not-know's
data = {'n':df.shape[0], 'm':4, 'k':2,
'y_pre':df['Vaccine Intent for %s (Pre)'%kind].values,
'y_see':[i%2+1 for i in df['Seen such online content'].values]} #"yes":2, "no":1 for ordinal logit
model = st.StanModel(model_code=model_code)
fit = model.sampling(data=data, iter=iters)
return fit
fit_filterbubble_T = model_filterbubble(df, group=1)
fit_filterbubble_C = model_filterbubble(df, group=0)
We can estimate the posterior distribution of those saying "yes", they had seen similar images online recently. This contributes to Table S8 of the paper.
stats_filterbubble_T = ut.stats_filterbubble(fit_filterbubble_T)
stats_filterbubble_C = ut.stats_filterbubble(fit_filterbubble_C)
ut.combine_dfs(stats_filterbubble_T, stats_filterbubble_C, '(Treatment)', '(Control)', perc=True)
It appears that indeed, as people become less accepting of the vaccine, they tend to respond more positively to having seen misinformation, and less to factual information. To see this better, we can compute the posterior estimates by contrasting with the baseline group of "Yes, definitely", and visualize with the helper function. This contributes to Table S8 and Figure S3 of the paper.
stats_filterbubble_T_delta = ut.stats_filterbubble(fit_filterbubble_T, contrast=True)
stats_filterbubble_C_delta = ut.stats_filterbubble(fit_filterbubble_C, contrast=True)
ut.combine_dfs(stats_filterbubble_T_delta, stats_filterbubble_C_delta, '(Treatment)', '(Control)', perc=True)
ut.plot_stats(100*stats_filterbubble_T_delta, 100*stats_filterbubble_C_delta, oddsratio=False, title='COVID-19 Vax Misinfo "Filter Bubbles" in the UK', title_l='Treatment', title_r='Control', xlab='% Change w.r.t. "Yes, definitely"', factor=0.5, titlesize=20, title_loc=0.4)
Therefore indeed, those indicating they would "definitely not" accept the vaccine were 10.2% more likely to have encountered similar misinformation, and 15.8% less likely to have encountered similar factual information online, when compared to those who would "definitely" accept the vaccine.